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‘ GROUP TESTING OF LISTENING COMPREHENSION 


ORAL PROFICIENCY 


Valerie Whiteson 


The matriculation examination of EFL. in Israel places the ^ 

* * 

•••• \ . ‘ 4 ’ 

greatest emphasis on reading skills 'an'd the least on writing skills,- 
Somewhere inbetv/een lie the oral skills. * 


Like many -other foreign language teachers elsewhere, Israeli.' 

teachers still treat- the written word with more respect, than the 

% 

spoken word. In an attempt t£ ensure that teachers would teach ^ 

conversation the Ministry' of Education introduced >a test of spoken 

English into the matric. exam. This test ha's been part *of the' 

exarft. for the past 12 or 13 yea?*s and it takes the form of a 

personal interview conducted by an examiner. The Ministry 

realises that such an interview cannot produce an objective 

evaluation of oral proficiency - they are, in fact, more interested 
h<xcku-><3L4K * 

in thn ^rmlrlabh^ nr the motivation it provides. Unfortunately 
it is still true to say that unless a particular skill is exam-, 
ined in the final examination it will not be taken seriously 
by both teachers and pupilsl This part of the examination is 
now an established fact and conversation forms part of every 
English high school course. 

Last year, for the first time, listening comprehension was 
introduced as an optional component of the exam. The authorities 
had carried out an experiment in nine different schools,ai.e. 

academic, vocational and rural. 237 pupils in the upper grades 

* , : 

took part. 3 passages were used, and the results proved to be 
satisfactory in that after the experiment 80^ of the pupils 
tested felt that listening comprehension' should form part of 
the final exam, and* 86$S wer<g in favour of including Listening 
Comprehension exercises’ in their English lessons. Teachers were 
also in* favour of including a listening test into the exam. 
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Last year, when Listening Comprehension was introduced for 
the first time as a option;oalyxfi§jaKEhaa± out of a population o£ 

* Id,000 candidates only 60 schools availed th,e?nselve& of the oppor— 
tunity. Schools were given the option of allowing their pupils to 


\ . 


answer two reading comprehension.passages spd or one reading oom- 

■ 5 

prehension passage and the listening comprehension. 

The form of the test 'was a passage of about 200 words which 

■\ *. . 
was broadcast over the radio. Each passage was spoken by an American 

* ' * 

female,and a British male. The passage was read 3 times in all. The 
pupils were required to ansWer m/c questions on the text. Although 


no statistics are available the general consensus of opinion is that' 
the reealts of the test were good and the Mininatry ha3 now decided 
that Listening. Comprehonsior^ will be an integral ..part of the 
exam. There aye no problems regarding th£ objectivity of marking 


m V 

tlie test but teachers still have to be encouraged to introduce’ 

' • % 

lls tening comprehension as a regular activity-in the classroom. 

Many of the teachers in Israel are new immigrants and* although ; 
a large number of these teachers come from English-speaking coun¬ 
tries there are many who do not. Unfortunately the proficiency, and 
particularly the oral proficiency, of some -of these teachers is laxx 
poor. In an attempt to standardise the required level of proficiency 
the Ministry of Education has introd^ed a national examination 
,for all teachers of EFL applying for a permanent teacher's license. 
This examination is also given by all Teacher Training Colleges 
and Universities which set and administer the examinations, /.themselves, 
after a central authority has passed thebi. The components of this 
•exam, are readings comprehension, written proficiency, grammar, 
listening comprehension and oral proficiency, 

I would like to speak to^y about these last TTo components. 
Everyone who has been involved in testing oral proficiency knows 
how difficult it is to be objective in evaluating performance. 
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At Bar IIan University the English dept. is responsible for *'■ 
setting the test for students who are planning to become^high-school., 

. * * A 

teachers. In the past we used an interview or role-playing' format 

for testing spoken English and we never thought it necessary to test 

• ‘ _ • ■ < 

listening Comprehension as all lectures are given in English in our 

* * <* *> - 

dept, in both literature .and linguistics. The test of oral proficiency 

* * 

was conducted by a panel of examiners with one of the examiners > 
doing the interview. This teacher was always so’involved in what 
she had to say that she couldn’t be objective at all. The other members 
of the panel found that fatigue quickly reduced their objectivity : 
too. It was also very difficult to provide each student with the 

same conditions. V/e chose a number of topics to discuss and although - 

i ' ■ 

we varied them as much as possible^we realized that students soon 
: • 1 ' 
found out about the topics and prepared,themselves in advance. Also 

. ' - * ’ ' • ' 
it was difficult to remeber the actual performance and we began to 

. u^a a tape-racorder to give us playback facilities. 1 This helped, 

but we knew that the only objectivity in our evaluation was the. fact 

that we usually had 3 or 4 examiners and if we all' agreed about a 

student’s performance we believed we were being fair. 

Most f.l. teachers are looking for a better.way of testing 

the oral skills and an article | "towards Objectivity in Group Oral 

Testing’* by David Folland and. Ddvid Robertson in- the Jan. 197& 

issue of SLT gave us the idea of a new method of testing. In their 

article Folland and Robertson describe their method of 'testing 

r» 

spoken English. V/e decided to adapt their method to our conditions^ -- 
. ' • % 
and combine the test of spoken English with a test of listening 

comprehension. * 

;<Ve examined a group of 27 students at the end of their second 
year BA in English Literature and Linguistics. Most of these s.tud- 

e' * » * 

ents are planning to become high-school English teachers. The reason 
We examine them at the end of their 2nd year is that if their results • 

’ i 

• t 






are not satisfactory they still have another yeay to improve their 


proficiency before graduating. We have combined this examination 

'j * - ■" 

f . - ■ . 

^ with the ^valuation required by the Ministry of Educatiorf for -future 
high-school teachers. Although the stress', in our department is very 


def lately on achievments in English kamnckgs literature and ling¬ 
uistics there is a final examination in; English^language which students 
are reqvfeired to pass befor^"*completing their.‘studies. Thin final exam. 

has stttdKJits made' students more aware that they must know language as f 

* * — ■ ~ . ■ 

well as literature and linguistics. ^ . 

.Our test of Aural/oral proficiency began in the language lab. The 
atudepts heard a lecture on the', "Study of Human Behavfo\^. ^-ring the 
second playing of the .xxpx lecture -they were required to answer a 
serias of M/C questions. to 'test their listening comprehension. The * 


test-was aimed at understanding of vocabulary in contexts ability to 


answer factual questions} the ability to see the main idea of a para- 

‘ . \ j - ' 4 

graph or-of a whele passage and toNdiscriminate between majpj* and 
minor ideas; the ability to understand cause and effect relationships^; 

to understand underlying presuppostion^ and to make" inferences and - 

• ‘ \ 

Vipally underloading of the speaker’s detentions. The M/C questions 

•V • i 

wa \ e given in written form and students were given ten minutes to read 

\ •• 


through the questions before the test began. .There were 25 questions 

\ ' 1 •r 


•* ' 4 • 

/add an examination of the results showed that the mean w? a ?2i6^ and 

* * • 
Ithe s.d. was 12.2. The range was 95 to 45. Bor many of the students 

the results of this part of the test were.disappointing. On the one hand' 

•V ■ ■•k .. ► 

the results, may be attributed to the fact that they had never’taken this 
typo of tost before; but oh the other hand it makes one woncer how much 
of their lectures they really do undorstahd. Israeli students invariable 


write down every word their lecturers say without mak ffg any attempt to 


develope the skills of note-taking. The kind of,test We'.gave them 

’ ■“ \ ^ ■ ■ , * - 

obliged them to analyse what they had. heard rax*hsxs£s* and.>eaoh 


conclusions. 

I 
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The. second part of the test concentrated on oral proficiency. » . 

We followed the advice of FallaaS end" Robertson when they suggest: that . 
the choice of a room it^whicb t6 hold the test is important. If must be' 
as unlike a cla V Voiin as possible in order to help'create a real-life 
situation. We held our test in the teachers common room during vacation. 
Groups of 5 students were invited to sit around a table in comfortable 
chairs and discuss the- lecture they had; just heard qn tape. As we had 
hoped, the topic of the leoture proved to be provocative and contro- 

veraial. _ • ' ' 

A panel of examiners cat the back of the room and listened to 

the discussion. </Bach teacher was responsible for evaluating a different 
aspect of oral proficiency for each student. In our department, in an 
attempt ,to try and achieve objectivity even in tests of written pro-, 
ficiency, the smo teacher marks the sane section of the exam, for 
each student e.g. one teacher marks the’cloze test; one the essay and 
so .on. In this test^out IfCtoMl score of 25 points we decided to 
award 4 for correct syntax; 4 for lexis; 2 for fluency and 2 for pro¬ 
nunciation; 4 for comprehension an* 9 for communicative competence. 

As these students are planning to become teachers the aip we set for 

them is near-native control of the language. 

) Each teacher gave a- marif for communicative competence which was 

an attempt to evaluate the overall performance . Then the average was 

taken as the final mark for that component. The students had been told ^ 

what we were listening for and we exj^ined that the mark for com- . 

munioative competence would be awarded for.their ability to inter-act 

with other members of the group. Anyone dominating the conversation 

would be penalized anfi bringing other people into the g^oup would be_ 

•* 

rewarded. The whole procedure waa not really novel' as part of their 
course in spoken English which preceded the Jc?owe included .this kind 
of activity except that in the classroom the teacher took part in the 

discussion. G * 
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We expected awkard initial pauaeK but there were none. The students 

• •••.•• ■>> . * “ - ' 

spoke naturally-and well, and in many cases they became so involved in 


the topic that it was clear they had forgotten they were being tested** 


ahey made ah effort to involve the .quiver meters of the-,^roup~and 


we 'hope that this will ba carr 


oyer into their everyday^onvershtional 
habits. We recorded the disoueslWvand stopped, after 15 minutes although' 


the students would hav% been 4^appy to go on. 


In« the original experi^Et the examiners used a series of plus and 
’tiinUfl signs to indicate major N ^nd minor errors*^7/e decided to change 
the^system and award the plus signs for excellence an choice ofj^pcab- r 

ular^j idiomatic use of'language; control of complex syntax etc. We 

\ * - ' 

chose to ignore minor errors tt which did hot lead to disruption of 

communication in any way. T*m referring to the kind of error made by 
native speakers in normal conversation. A minus was given for the kind 
of error which did interrupt communication such as wrong choice of 


vocabulary; wrong ten3e and certain pronunciation errors. . 

<* * ' * 


After each group concluded their discussion it was quite oasy to 
award markg v for each component by referring to the plus and minus signs 
made for each student during the test.jln a number of cases-when the 
teacher's evaluation was challenged by another member of the team we 
replayed the discussion ana invariably the teacher who had concentrated 

J • # ' . 

on that aspect of the performance was feund to be correct. It was also 


comparatively easy to arrive at a 'composite grade for communicative 

* 

competence. The grades were then totalled and multiplied by 4 to arrive 


at a percent age. The’final grades v^e re tberi correliated with the indiv- 

^measured by Sn ^rrrin »<3 rnnk 

idual's grade for the year's worx. The'correlation was .861 After the 


examination the students were advised what weaknesses had shown up 

during thrb tost. We hadn’t realised that the teat would also prove to 

% 

* 

be diagnostic. ' * • 

In previous years we had allocated ten minutes to a&i student for 
the interview so that this method of testing proved to be less .time- 
consuming. The test took about one third of the time.. The students 



appreciated the f^ee validity of the teat ; in that they agree that the •. 

testing situation was as near to a real life situation as they could „ 

' * • • 9 ''\ 
v • ' V • - • n • 

posmiblay expect in a testing situation. The panel of examiners felt 
that their evaluation of the stidents* performances was as objective as 
possible particularly-as they felt leas tired as the test, was so much 
shorter. We also found the experience very rewarding. Hearing our . 
students talking to each other'in a manner that would be acceptable 

i • . _ . 

in any Snglisn-spesking society was, rewarding. On the whole the grades 
■were higher than in the listening comprehension.The mean was 82$ and- 
the a.d. was 11.3. The rangd was 96'to 50. ' , 

. On the basis of"this one test we could not reach many conclusions \ 

, " * - ' • • v 

but we found it hard to believe that our students could speak better .* 
than they could understand. A more recent listening comprehension test^ 

without the spoken element yielded-a similar result.-The mean was 73.4 

* ■* • . 4 

and the s.d. 11.9. The"correlation between the results of the tests 
measured by the Spearman's rank order correlation was .65 which is • 

pOsit,iv^'out not as high as v/e had expected.. This seems to indicate 

* - 

that listening and speaking are different skills and should be measured 

v •> ... . * 

separately in order to provide an overall estimation of aural/oral skill! 

• x ‘ ■*’ ' •' ' . ' k 

.Ye feel, as Folland and Robertson do, that this group test -and.. ‘ 

’ - ' o . 

marking system can be adapted^and used in other situations and for 

f 

otner languages including.mother tongue testing. 'Ye would like to see 
it used in schools at a lower level as well. The techniques involved * 

in this kind of; teat are well worth cultivating in order to develop 

inter-personal relationships and to further our goals of encouraging' 
communication. 
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